#Python's process pool

Creating and destroying processes repeatedly incurs overhead. When processes are created and terminated frequently, program performance can degrade.

To address this, we often reuse processes by suspending them instead of destroying them. The next time a process is needed, it resumes execution with new code.

This group of managed reusable processes is called a process pool. In Python, you can create one using the Pool class from the multiprocessing module. The constructor accepts the number of worker processes.

Example:

from multiprocessing import Pool
import os

def worker(i: int):
    print(f"PID:{os.getpid()} : {i}")

if __name__ == '__main__':
    pool = Pool(4)                              # Create a process pool with 4 workers
    for i in range(100):
        pool.apply_async(worker, args=(i, ))    # Submit task to the pool
    pool.close()                                # No more tasks will be submitted
    pool.join()                                 # Wait for all worker processes to finish

Output:

• The scheduling of processes is nondeterministic, so the output order is not guaranteed.

PID:109516 : 0
PID:109516 : 1
PID:109516 : 2
PID:109516 : 3
PID:109516 : 4
...
PID:56556 : 97
PID:109516 : 98
PID:56556 : 99